Dynamic claims submission system

ABSTRACT

Embodiments are directed to a method for determining an interview script in a claims submission. The method may comprising receiving data relating to a claim being submitted, which may include claims submission data input by a user, information relating to the user, and one or more features. Data associated with the one or more features may be determined from an artificial intelligence model. A first score based on the data associated with the one or more features and data associated with the information relating to the user may be determined and used to determine an interview script. In one embodiment, questions in the interview script may continue to be provided to the interviewer computer if a continually updated score remains above a predetermined threshold. In another embodiment, the user may be routed to a live interview with a human representative if a continually updated score drops below a predetermined threshold.

CROSS-REFERENCES TO RELATED APPLICATIONS

None.

BACKGROUND

In today's technological environment, companies have various systems forhandling complaints, reports, or other claims made by individuals. Toprocess these claims, these companies often allow individuals to connectto their system over the internet or by phone. Human representatives maybe trained as to how to interact with the individuals, and sometimes therepresentatives will be given a set of instructions for differentscenarios, such as in the form of a script.

A variety of technical issues may arise from a claims submission system.One such issue is in the case of disingenuous reporting. It is notuncommon for criminals to find ways to game the system in order to makea profit. For example, a criminal may learn of a company's specificprotocol for handling various claims, and may figure out how to answerspecific questions in order to receive money in the form of statementcredits or in insurance payments. With a static predetermined script,this may be relatively easy to accomplish. Another issue is in the caseof human error. For claims submission systems handled over the internet,a poorly designed user interface may cause a user to mistakenly type inthe wrong information. What is needed in the art is a method forautomating the claims submission process in a manner that cannot beeasily gamed or abused, and that allows for inaccurate information to becorrected.

Embodiments of the invention address these and other problems,individually and collectively.

BRIEF SUMMARY

Various embodiments are directed to a method for determining aninterview script in a claims submission. The method may comprisingreceiving data relating to a claim being submitted, which may includeclaims submission data input by a user, information relating to theuser, and one or more features. The method may further comprise storingthe data relating to the claim submission in a database and retrievingfrom the database, data associated with the one or more features asdetermined from an artificial intelligence model. A first score based onthe data associated with the one or more features and data associatedwith the information relating to the user may be determined. Aninterview script may be determined based at least upon the first scoreand a first question in the interview script may be provided to aninterviewer computer. A response to the first question may be received,and a second score based at least upon the data in the database and theresponse to the first question may be generated. The interview scriptmay be updated based at least upon the second score, and a secondquestion in the interview script may be provided to the interviewercomputer, based at least in part upon the second score.

In one embodiment, questions in the interview script may continue to beprovided to the interviewer computer if a continually updated scoreremains above a predetermined threshold.

In another embodiment, the user may be routed to a live interview with ahuman representative, if a continually updated score drops below apredetermined threshold.

Other embodiments are directed to systems, server computers, clientcomputers, portable consumer devices, and computer readable mediaassociated with methods described herein.

A better understanding of the nature and advantages of embodiments ofthe present invention may be gained with reference to the followingdetailed description and the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a high-level diagram depicting a process for training andusing a machine learning model.

FIG. 2 shows a depiction of a system for conducting an interviewaccording to an embodiment of the invention.

FIG. 3 shows a block diagram of a server computer according to anembodiment of the invention.

FIG. 4 shows a process flow diagram for conducting an interviewaccording to an embodiment of the invention.

FIG. 5 shows a flowchart for a method for generating a dynamic interviewscript according to an embodiment of the invention.

FIG. 6 shows a depiction of a data flow diagram for an automated claimsubmission system according to an embodiment of the invention.

FIG. 7 shows a data flow diagram for building an artificial intelligencemodel for generating an interview script according to an embodiment ofthe invention.

FIG. 8 shows an example of a path of nodes in a concept graph accordingto an embodiment of the invention.

FIG. 9 shows an example of a conceptual graph.

TERMS

The term “artificial intelligence model” or “Al model” may refer to amodel that may be used to predict outcomes in order achieve apre-defined goal. The Al model may be developed using a learningalgorithm, in which training data is classified based on known orinferred patterns. An Al model may also be referred to as a “machinelearning model” or “predictive model.”

A “feature” may refer to a property of data that may be used torecognize patterns. An input feature may be data that identifiable by anartificial intelligence for making predictions, as learned throughtraining. An input feature may be identified as a collection of one ormore input nodes in a graph, such as a path comprising the input nodes.For example, an input feature may be identified as a path (set)comprising input nodes for ‘Cardholder: Rick’, ‘claims lost/stolencredit card’ and ‘text copy and pasted.’ Features may be organized intoa searchable format, such as a graph database or index table. Forexample, input features may be organized as a list of keys that may beindexed, queried, and assigned values for making predictions. When afeature is detected data, an Al model may be used to predict dataassociated with the feature, such as a predicted riskiness orprobability of fraud associated with the feature.

A “conceptual graph” may refer to a graph representation for logic basedon the semantic networks of artificial intelligence. A conceptual graphmay comprise concepts and conceptual relations for forming a sentence.In a conceptual graph, rectangular nodes may representing concepts andcircular nodes may represent conceptual relations. For example, withreference to FIG. 10 , a conceptual graph may comprise concept 901(Go)which may be the verb of a sentence. Concept 901 may further be linkedto concept 905 (Person: Rick), concept 906 (City: Austin), and concept907 (Train), which may be nouns linked to “Go” through conceptualrelations 902 (Agent), 903 (Destination), and 904 (Instrument)respectively. The conceptual graph may then be used to form thesentence, “Rick is going to Austin by train.”

A “concept” may refer to a word or phrase for describing a general idea.For example, an interview for a claims submission may be described bythe concepts: ‘Cardholder: Rick’, ‘Lost/Stolen Credit Card’, and ‘HighRisk.’ Concepts may be expressed in a topological graph as nodes. In aconceptual graph, concepts may be represented by rectangles.

A “conceptual relation” may refer to a word or phrase for relatingconcepts. For example, the concept, ‘Cardholder: Rick’ may be linked tothe concept, ‘High Risk’ through the conceptual relation, ‘Is.’ Theconcept, ‘Cardholder: Rick’ may further be linked to the concept,‘Lost/Stolen Credit Card’ through the conceptual relation, ‘Claims.’

A “claim submission” may refer to a request for something that is owed.For example, a claim submission may be a submission of an insuranceclaim, in which a user requests, per an insurance policy between theuser and an insurance provider, an owed imbursement of funds. In anotherexample, a claim submission may be a request for an reimbursement offunds related to fraudulent transactions posted to one's purchasingaccount. In yet another example, a claim submission may be a request formaintenance/repair services that a service provider may grant to itscustomers.

The term “language parsing” or “linguistic parsing” may refer to theanalysis of a string of symbols to recognize words and phrases in asentence and separate them such that their relationships and meaningsmay be understood. A linguistic parser may be used to recognize motifsin submitted text, such as motifs related to text that is frequentlysubmitted in fraudulent claims. An example of a linguistic parser mayinclude The Stanford Natural Language Parser.

A “semantic decision table” may refer to a decision table in whichdecision rules are mapped to concepts. The concepts may be concepts froma conceptual graph, and the decision rules may correspond to rules forforming an interview question. Conceptual relations for determiningrules in a semantic decision table may be expressed as binary factsreferred to as “lexons.” More information regarding semantic decisiontables can found at:

-   -   Tang Y., Meersman R. (2007) On Constructing Semantic Decision        Tables. In: Wagner R., Revell N., Pernul G. (eds) Database and        Expert Systems Applications. DEXA 2007. Lecture Notes in        Computer Science, vol 4653. Springer, Berlin, Heidelberg.

The term “common logic interchange format” may refer to a standard forspecifying common logic. “Common Logic” may refer to a framework for afamily of logic languages intended to facilitate the exchange andtransmission of knowledge in computer-based systems. The ISO standardfor common logic is described in ISO/IEC 24707:2007. More informationcan be found at:

-   -   https://www.iso.org/standard/39175.html.

The term “ant colony optimization” may refer to a probabilistictechnique for solving computational problems which can be reduced tofinding shortest paths through graphs. Ant colony optimizationalgorithms may include algorithms in which path information iscommunicated to a plurality of computational agents between subsequentiterations.

A “topological graph” may refer to a representation of a graph in aplane of distinct vertices connected by edges. The distinct vertices ina topological graph may be referred to as “nodes.” Each node mayrepresent specific information for an event or may represent specificinformation for a profile of an entity or object. The nodes may berelated to one another by a set of edges, E. An “edge” may be describedas an unordered pair composed of two nodes as a subset of the graphG=(V, E), where is G is a graph comprising a set V of vertices (nodes)connected by a set of edges E. For example, a topological graph mayrepresent a transaction network in which a node representing a financialtransaction may be connected by edges to one or more nodes that arerelated to the transaction, such as nodes representing information of adevice, a merchant, a transaction type, etc. An edge may be associatedwith a numerical value, referred to as a “weight”, that may be assignedto the pairwise connection between the two nodes. The edge weight may beidentified as a strength of connectivity between two nodes and/or may berelated to a cost or distance, as it often represents a quantity that isrequired to move from one node to the next.

A “sequence graph” or “graphic sequence” may refer to a sequence ofnumbers which can be the degree sequence of a graph. For an undirectedgraph, a “degree sequence” may refer to a sequence of nodes in a graph.For example, a degree sequence may identify a path of nodes forestablishing rules for an Al model.

DETAILED DESCRIPTION

Embodiments of the invention are directed to a method for determining aninterview script. Embodiments described in the detailed descriptionherein may comprise a method that uses an artificial intelligence (AI)model to generate an interview script based on features of a submittedclaim. The AI model may provide questions to an interviewer computer,such that only accurate information may be included in a processedclaim. This may be done by providing follow-up questions that may assistin clarifying incorrectly entered information. In the instance of afalse claim being submitted, questions may be continually asked untilthe submitter of the false claim quits the interview. Features used toassess the risk of a potentially fraudulent claim or a claim containingpotentially inaccurate information may include data associated with userinformation and data associated with text inputted by the user, asidentified through patterns learned by an Al. The Al may continuouslyadapt to new features associated with a disingenuous claim submission,and may dynamically update the interview script such that a criminalactor may not easily game the claim submission system.

I. Overview of Invention

Embodiments of the invention are directed to a system for generating adynamic interview script using artificial intelligence. FIG. 1 shows ahigh-level diagram depicting a process for training and using a machinelearning model. Process 100 starts with training data, shown as existingrecords 110. The training data can comprise various data samples, whereeach data sample includes input data and known output data. For example,training data may be aggregated from a variety of sources acrossmultiple networks, such as transaction networks and social medianetworks. The input data can be profile data of a user (e.g. a user'stransaction records or social media data), and the output data can be aclassification of their behavior (e.g. high risk individual or low riskindividual).

After training data is obtained, a learning process can be used to trainthe model. Learning module 120 is shown receiving existing records 110and providing model 130 after training has been performed. As datasamples include outputs known to correspond to specific inputs, a modelcan learn the type of inputs that correspond to which outputs. Oncemodel 130 has been trained, it can be used to predict the output for anew request 140 that includes new inputs. For instance, model 130 may bea model that can predict whether a user is lying based on features ofthe user's communication. Model 130 is shown providing a predictedoutput 150 based on new request 140. Predictive output 150 may be anyoutput that is predicted to achieve a desired result, such as a phraseor question that is predicted to deter a criminal. In this manner, thewealth of the training data can be used to create artificialintelligence that can be advantageously used for a particular problem.As explained above, the invention described herein utilizes anartificial intelligence model for the purpose of generating a dynamicinterview script. The interview script may be used, for example,processing a claim such an insurance claim, or may be used for anyinteraction in which an interviewee is requiring assistance with aparticular matter.

FIG. 2 shows a depiction of a system for conducting an interviewaccording to an embodiment of the invention. System 200 may compriseuser device 202, interviewer computer 210, and server computer 220. Theuser device 202 may be operated by a user 201 and may be incommunication with the interviewer computer 210. The interviewercomputer 210 may receive back-end support from one or more servercomputer(s) such as server computer 220. In addition, server computer220 may be coupled to or have access to one or more sources of data(i.e. databases) such as from aggregate data database 230, short-termhistory 240, and graph database 250.

According to embodiments of the invention, user 201 may use user device202 to conduct an interview with interviewer computer 210 (e.g. such asfor a claims submission or for reporting an incident or broken device).This may be done by establishing a channel of communication withinterviewer computer 210 over a network 270 such as the internet ortelecommunications network. User device 202 may be any device forsending information to interviewer computer 210 over network 270, suchas a personal computing device, laptop, mobile communications device,telephone, etc. User device 202 may be used to send messages that may bereceived and processed by interviewer computer 210 in the form of data.For example, an input device of user device 202 such as a keyboard maybe used to generate text that may be sent as data to interviewercomputer 210. In another example, user device 202 may be used to sendvoice messages over a telecommunications network, which interviewercomputer 210 may receive and process into data. In yet another, userdevice 202 may be used to take a picture and generate an image file thatmay be sent to interviewer computer 210.

Messages received by interviewer computer 210 from user device 202 maycomprise data relating to a claim submission. A claim submission may beany request for something that is owed. For example, a claim submissionmay be a submission of an insurance claim, in which a user requests, peran insurance policy between the user and an insurance provider, an owedimbursement of funds. In another example, a claim submission may be arequest for an reimbursement of funds related to fraudulent transactionsposted to one's purchasing account. In yet another example, a claimsubmission may be a request for maintenance/repair services that aservice provider may grant to its customers.

The data relating to the claim submission may be sent from interviewercomputer to server computer 220. Server computer 220 may be any servercomputer for receiving, processing, storing, retrieving, and sendingdata. For example, server computer 220 may be a server computer of atransaction processing network (e.g. Visanet) that may receive, process,store, retrieve, and send data relating to transactions. In oneembodiment, server computer 220 may be a server computer that receives,processes, stores, retrieves, and sends data to a client computer in aclient-server model. The client computer may be, for example, aninterviewer computer. In another embodiment, server computer 220 may bea server computer that receives, processes, stores, retrieves, and sendsdata in a cloud-based model.

Server computer 220 may store, update, and retrieve data from one ormore databases such as, aggregate data database 230, short-term historydatabase 240, and graph database 250. The data in the one or moredatabases may be data used by server computer 220 to generate, and makepredictions using, an artificial intelligence (AI) model. The AI modelmay run on server computer 220, and may be used to predict appropriateresponses for achieving a predetermined goal. In one embodiment of theinvention, the predetermined goal may be to prevent false or inaccurateinformation from being included in a processed claim. This may be done,for example, by predicting and generating responses that may deter acriminal actor from continuing a claims submission or generatingresponses that clarify information that is frequently enteredincorrectly.

Aggregate data database 230 may store aggregate data that is sharedacross multiple networks. The aggregate data may be accessed by servercomputer 220 and used as training data for an AI model. For example, theaggregate data may be collected in a transaction network and used togenerate predictive models relating to the behavior of the transactingparties. Examples of aggregate data may include fraud data, real-timedata feeds, and other outside data. Fraud data may comprise dataregarding fraudulent transactions, such as negative lists linkingtransaction identities (e.g. account numbers, user IDs, device IDs,etc.) to reported instances of fraud. Real-time data feeds may includedata that is received over a network in real-time such as data fortransactions being conducted. Other outside data may include any otherdata that can be extracted and used to make predictions, such as censusdata, financial data, social media data, etc. In one embodiment, theaggregate data may be data that is stored and processed across multiplecomputers according to the Apache Hadoop framework.

Short-term history 240 may comprise short-term data that can be used formaking current predictions. Short-term data may include prior requeststo an AI model (i.e. historical requests for a prediction), which may beused to detect features present in information and predict a presentoutcome. For example, short-term data may include a history of messagessent between user device 202 and interviewer computer 210 or othermessages sent between other user devices and other interviewercomputers. The history of messages may be used to determine anappropriate response to a current message sent from user device 202 tointerviewer computer 210 (e.g. a follow-up question to an answersubmitted by user 201).

Graph database 250 may comprise graph data for generating an interviewscript. The graph data may include a plurality of conceptual graphs forrepresenting knowledge and reasoning. The plurality of conceptual graphsmay be used to represent concepts used in a semantic network for an AImodel. For example, graph data may comprise a plurality of conceptualgraphs representing an appropriate combination of words and phrases forforming a sentence. The formed sentence may be used to generate aresponse to user 201 during an interview, as determined using an AImodel running on server computer 220 and as delivered by interviewercomputer 210.

According to embodiments of the invention, data received and/orretrieved by server computer 220 may be used to build an AI model formaking predictions. The AI model may be used to determine responses touser 201 in an interview (e.g. claims submission). The generatedresponses may be responses that are predicted to assist in identifyinginaccurate information. For example, an appropriate response predictedby an AI model may be a follow-up question that clarifies if user 201has correctly inputted text into user device 202 (e.g. “You live inTexas, is that correct?”). In another example, a response may be afollow-up question that reveals if user 201 is likely giving falseinformation (e.g. “When did you first notice that your card was lost orstolen?”).

Server computer 220 may use aggregate data in aggregate data database230 as training data for building and updating the AI model. Forexample, data relating to interview questions, user responses, previousoutcomes of previous interviews, and external data (e.g. fraud data) maybe used to identify patterns. This may be done, for example, using alearning algorithm to score a plurality of interview questions and userresponses for their predictiveness of desired outcomes (e.g. scoringinterview questions higher if they deter a fraudulent individual orgiving a high risk score to user responses linked to known incidences offraudulent claims). Specific types of information that can be scored forpredictiveness and used to train and generate a better fitting model maybe referred to as features. For example, a feature may be the speed atwhich a question is responded to by a user, and the feature may bescored for its relation to a high probability of fraud (e.g. a questionthat is responded to in less than 1 second may be indicative offraudulent behavior and may receive a high score). If in a futurerequest to the AI model a high scoring feature is detected (e.g.response time less than 1 second), then the AI model can make aprediction based on the feature due to its known correlation to aspecific outcome (e.g. high probability of fraud).

During an interview, messages may be sent between user device 202 andinterviewer computer 210, and may comprise data that may be sent toserver computer 220. The data may be stored in short-term history 240and may be data relating to a claim submission. The data relating to theclaim submission may include data input by user 201 (e.g. text),information relating to user 201 (e.g. name, account number, device ID,IP address, etc.), and one or more features (e.g. risk features). Theone or more features may be predictive features that may be identifiedand used in an AI model for making predictions. In one embodiment, theone or more features may include a time of day, a claim type, a methodof entry, and/or an average time to respond. In another embodiment, theone or more features may also include features of text inputted by theuser. For example, a natural language parser may be used to parse asentence and determine features that are indicative of certain behavior,such as lying.

The one or more features may be used by an AI model running on servercomputer 220 to determine data associated with the one or more features.For example, an AI model running on server computer 220 may compare theone or more features to an index table of predicted outcomes (e.g.associated risk of each feature), as learned through training. Servercomputer 220 may further determine data associated with the informationrelating to user 201. For example, server computer 220 may retrieve,from aggregate data database 230, data linking user 201's account numberto high risk behavior (e.g. identified fraud or low credit score).

The data determined from the AI model and the data associated with user201 may then be used by the server computer to determine a first scorefor the interview. The first score may be, for example, a risk scoreassessing the currently evaluated risk of user 201 being disingenuous orof the information submitted by user 201 being inaccurate or false. Therisk score may be stored by server computer 220 in short-term history240 and may be used to update the AI model. For example, the risk scoremay be used to score the predictiveness of the one or more features sothat future predictions made by the AI model may be more accurate.

Server computer 220 may then determine an interview script based atleast upon the first score. In one embodiment, server computer 220 mayquery graph database 250 for a conceptual graph based on the firstscore, data associated with user 201, and data associated withinformation entered by user 201 (e.g. a conceptual graph comprisingwords of user entered text or features thereof). For example, servercomputer 220 may determine that a user identified as ‘Cardholder: Morty’submitted a response containing the word, “Singapore” in less than asecond. An AI model may be used to determine that the response isassociated with a risk score of 90 out of 100. Server computer 220 maythen generate a graph query comprising, ‘Cardholder: Morty’,‘Singapore’, and ‘risk score =70 or higher.’ The graph query may beperformed to retrieve a conceptual graph from graph database 250 for aconceptual graph comprising nodes for ‘Cardholder: Morty’ ‘Singapore’and ‘risk score=70 or higher.’ The conceptual graph may comprise nodesfor concepts (e.g. ‘Morty’ ‘Singapore’, ‘risk score=70 or higher’)linked together through conceptual relations. For example, the conceptsmay be linked by the conceptual relations: ‘Agent’, ‘lives’, ‘is’, and‘initiate,’ which may derive meaning for initiating the decision: ‘Mortylives in Singapore and is high risk [risk score>70]. Initiate scriptA:High Risk.’ An interview script may then be determined from theconceptual graph (e.g. ‘script A:High Risk), and a follow-up questionfrom the interview script may be sent to user device 202 (e.g. “wouldyou like to be continue the interview in Mandarin?”).

According to embodiments, a user response to a follow-up question from adetermined interview script may be evaluated to determine a second scorefor the interview. For example, an interview script determined for aphone interview may comprise the question, “what is your mother's maidenname?” The user may respond instantly, and in a manner that isdetermined to be low risk (e.g. an AI model for predicting the risk of alying user based on features of audio data may determine a low riskscore). The features of the user's response may change the score for theinterview, and the second score for the interview may result in adifferent query. For example, a second graph query may be performed inwhich a conceptual graph may comprise nodes for initiating the decision:‘Morty is low risk. Initiate script A: low risk.’

The determine interview script may then be updated based on the graphquery, and a second question from the interview script may be sent tothe user (e.g. “Your mother's maiden name is ‘Smith,’ is that correct?”)

In one embodiment, a determined interview script may be generated suchthat an individual identified as potentially giving false or inaccurateinformation may not be allowed to submit a claim for processing. Theautomated questions of the determined interview script may be generatedsuch that interviews scoring higher with regards to inaccuracy mayreceive more aggressive questioning, such as to extract more informationfrom user 201. In one embodiment, automated questions/responses may becontinually sent to user 201, until user 201 terminates the claimsubmission or submits responses that lower the risk score for theinterview below a predetermined threshold (e.g. by providing accurateinformation). In another embodiment, individuals identified as being lowrisk or providing accurate information may be routed to a live agent(i.e. a human interviewer) who may process the submitted claim.

FIG. 3 shows a block diagram of a server computer according to anembodiment of the invention. Server computer 300A may be any servercomputer for receiving, processing, and sending data according to theembodiments of the invention, such as server computer 220 of FIG. 2 .Server computer 300A may comprise processor 310, network interface 320,and computer readable medium 230. Computer readable medium 230 maycomprise one or more modules of code that may be used to carry outembodiments of the invention. For example, computer readable medium 230may comprise communication module 330A, linguistic parsing module 330B,entry method analysis module 330C, user history lookup module 330D, riskdetermination module 330E, learning module 330F, graph query module330G, script determination module 330H, live interview routing module330I, audio file analysis module 330I, image file analysis module 330K,and video file analysis module 330L. Server computer 200A may further becoupled to one or more databases such as aggregate data database 300B,short-term history 300C, and graph database 300D, which may correspondto aggregate data database 230, short-term history 240, and graphdatabase 250 of FIG. 2 respectively.

Communication module 330A may comprise code for instructing processor310 to receive, send, and reformat messages that may be received overnetwork interface 320. For example, communication module may comprisecode for receiving and sending messages over a network (e.g. network 270of FIG. 2 ) during an interview process, such as during a claimsubmission.

Linguistic parsing module 330B may comprise code for instructingprocessor 310 to analyze a string of symbols. For example, the string ofsymbols may be text received in a message and linguistic parsing module330 may comprise code for recognizing words and phrases in a sentenceand separate them to understand their relationships and meanings. In oneembodiment, linguistic parsing module 330B may comprise code for anatural language parser or statistical parser such as The StanfordNatural Language Parser.

Entry method analysis module 330C may comprise code for instructingprocessor 310 to determine information relating to how received data wasoriginally entered by a user. In one embodiment, entry method analysismodule may comprise code for determining how quickly text data wasentered, determining if inputted text had been copy and pasted, and/ordetermining the length of pauses between typed information. For example,entry method analysis module may be used to determine that user 201 ofFIG. 2 answered an interview question by quickly copy and pasting apredetermined response stored on user device 202, and this feature maylater be identified as a strong predictor of a fraudulent claimsubmission.

User history lookup module 330D may comprise code for instructingprocessor 310 to lookup data associated with information relating to auser. For example, user history lookup module 330D may comprise code fordetermining a user ID or a device ID and other user informationextracted from data received in a message (e.g. data relating to a claimsubmission), and may further comprise code for querying aggregate datadatabase 300B for data linked to the user ID or device ID. The data mayinclude a history of behavior relating to the user such as a history ofinterviews (e.g. history of claim submissions), known instances offraud, or reports of the user's information being stolen.

Score determination module 330E may comprise code for instructingprocessor 310 to determine a score associated with received data. In oneembodiment, the score may be a score associated with a user response inan interview or for an interview itself. The score may be determinedbased on one or more features and data associated with a user and his orher responses. For example, score determination module 330E may comprisecode for recognizing one or more risk features associated with textinput by a user in a claims submission. Score determination module 330Emay further comprise code for determining a risk score based on the oneor more risk features. According to embodiments of the invention,examples of risk features may include a time of day a message is sent, aclaim type, a method of entry for received data, and/or an average timeto respond (i.e. how quickly data was entered after an interviewquestion has been asked).

Learning module 330F may comprise code for instructing processor 310 totrain an AI model for making predictions about received data. Forexample, learning module 310 may comprise a machine learning algorithmthat may be used to generate and update an AI model based on data storedin a database. The AI model may be used to make predictions based oninput data. For example, the predictive model may be used to predict anappropriate follow-up interview question based on a response from a user(e.g. based on text in a claims submission).

Graph query module 330G may comprise code for instructing processor 310to perform a query for a graph stored in a graph database, such asconceptual graph data database 300C. For example, graph query module330G may comprise code for querying graph data database 300C for aconceptual graph linked to a specific risk score and/or set of text. Thespecific risk score and set of text may be information relating to aclaims submission (e.g. a set of text inputted by a user). In oneembodiment, the graph that is to be queried by graph query module 330Gmay be used to determine binary facts, or “lexons” in a semanticdecision table for driving an interview script. For example, the graphmay comprise relationships between risk scores and concepts for aninterview (i.e. words or phrases), which may be used to determineappropriate paths for the interview, as defined by a prebuilt semanticdecision table comprising claim submission rules.

Script determination module 330H may comprise code for instructingprocessor 310 to determine a script based on a queried graph. Forexample script determination module 330H may comprise code fordetermining a set of text associated with information relating to aconceptual graph. The information relating to the conceptual graph maycomprise concepts and conceptual relations that may be used to form asentence. In one embodiment, the information relating to the conceptualgraph may be information in a semantic decision table, such as lexonsreflecting rules for different sets of interview concepts.

Live interview routing module 3301 may comprise code for instructingprocessor 310 to connect a user device to a human interviewer. Forexample, live interview routing 3301 may comprise code for routing auser device (e.g. user device 202 of FIG. 2 ) from connecting with aninterviewer computer running an automated script (e.g. interviewercomputer 210 of FIG. 2 ) to an interviewer computer or call center fromwhich they may speak to a representative that may process a claim. Inone embodiment, live interview routing module 3301 may be initiated ifit is determined that a submitted claim is genuine (e.g. very low riskscore).

Audio file analysis module 330J may comprise code for instructingprocessor 310 to analyze features of an audio file. For example, audiofile analysis module 330J may comprise code for detecting a long pausein a response or may detect an inflection in a user's voice. Thefeatures of the audio file may be any pattern that may be repeatedlyrecognized by an AI model in data and that may be indicative of certainbehavior. For example, an audio file for a claims submission in which auser's voice is relatively monotone may be indicative of a fraudulentclaim. In another example, an audio file for a user reporting anincident in which the user's voice is shaky or hurried may be indicativeof a user who is sincere in reporting a real incident.

Image file analysis module 330K may comprise code for instructingprocessor 310 to analyze features of an image file. For example, imagefile analysis module 330K may comprise code for recognizing facialfeatures to recognize a user. In other examples, image file analysismodule 330K may comprise code for recognizing features of objects thatmay be related to a submitted claim, such as features of credit cards,bar codes, packages, vehicles, products, damaged body tissue, etc. Thefeatures of the image file may be any pattern that may be repeatedlyrecognized by an AI model in data and that may contain more informationfor making better predictions. For example, an image file for a photo ofa car accident may comprise a specific amount of light as a feature,which may indicate the time of day the photo was taken or may beindicative of some other pattern such as a certain level of risk for aclaims submission. In another example, an image file may contain aslight nuance in color or contrast that may be a feature of photos takenwith a specific type or manufacture of camera lens which may further beassociated with a certain level of risk in a claims submission.

Video file analysis module 330L may comprise code for instructingprocessor 310 to analyze features of a video file. For example, videofile analysis module 330L may comprise code for recognizing features ofa video that is the same or similar to a video submitted for afraudulent claim. The features of the video file may be any pattern thatmay be recognized by an AI model and that may aid in making betterpredictions. For example, a video length or file size may be a featurethat is indicative of whether a submitted claim is fraudulent orgenuine.

According to embodiments of the invention, server computer 300A mayprovide back-end support for an interview computer, such as interviewercomputer 210 of FIG. 2 . The server computer 300A may be used togenerate an automated interview script comprising responses that arepredicted to assist in identifying inaccurate or false information. Thescript may further be a script comprising questions predicted to detercriminal actors/prevent inaccurate or false information from beingincluded in a processed claim.

FIG. 4 shows a process flow diagram for conducting an interviewaccording to an embodiment of the invention. According to embodiments ofthe invention, an interview may be conducted between a user device (e.g.user device 202 of FIG. 2 ) and an interviewer computer (e.g.interviewer computer 210 of FIG. 2 ). The interview computer may baseits responses to a user of the user device (e.g. user 201 of FIG. 1 ) onan interview script that can be determined using an AI model.

Process 400 may begin at step S401, in which an interview between a userdevice and interviewer computer may be started. The interview may beinitiated by a user that establishes a connection with an interviewsystem (e.g. a claim submission system). The connection may beestablished by establishing a connection between a user device and aninterviewer computer over a network. For example, the user may use theuser device to connect to the internet and may establish a connectionwith the interview system by accessing a website from a browser. Inanother example, the user device may be a phone, and the user mayestablish a connection with the interview system over atelecommunications network. Upon connecting with the interview system,the user may begin an interview with the interviewer computer.

At step S402, the user may submit initial data using the user device.The initial data may be data relating to a claims submission, which mayinclude text, audio, images, or video submitted by the user. Forexample, the interview system may provide a user interface that may bedisplayed on the user's device and allow the user to input text. Theinitial data may be sent to the interviewer computer and forwarded to aserver computer (e.g. server computer 300A of FIG. 3 ). The text mayinclude a description of the claim that is being submitted or incidentthat is being reported. For example, the user may use an input device(e.g. keyboard) of the user device to input text for, “Hi, I would liketo report a stolen credit card.” In another example, the user maycontact the interview system via phone. The user may submit informationfor a claim by voice and the submitted audio may be processed into data.In yet another example, the user may send a Short Message Service (SMS)text message to the interview system to initiate the claims submissionprocess using his or her phone. Other examples of data relating to aclaims submission may include image data (e.g. a selfie of the user, aphoto of a credit card, an image of a damaged vehicle, etc.) or videodata (e.g. a video of an incident that the user wishes to report or avideo of the user explaining an incident).

At step S403, user device data may be collected. In one embodiment, theuser device data may be collected by analyzing the header of a message.For example, the interviewer computer may receive a data message fromthe user device in which a device ID and/or IP address of the userdevice is written into a header of the message. In another embodiment,the user device may be a phone, and the user device data may comprise atelephone number and a location of the phone. For example, theinterviewer computer may receive an SMS or Multimedia Messaging Service(MMS) text message comprising a user data header (UDH) at the start ofthe message. According to embodiment, the user device data may bereceived by the interviewer computer and forwarded to the servercomputer.

At step S404, the initial data entered by the user and the collecteduser device data may be stored in short-term history. The short-termhistory may be a database comprising data that may be used to determinean interview script. For example, the short-term history may comprisedata for the current conversation between the user device andinterviewer computer. The data may comprise features that may be used asinputs to an AI model. The AI model may be an AI model trained topredict appropriate responses that assist in identifying accurate andinaccurate information in a claims submission.

At step S405, the server computer may determine one or more featuresfrom the data relating to the claim submission and may retrieve dataassociated with the one or more features from a database (e.g. aggregatedata database 230 of FIG. 2 ). For example, the server computer mayanalyze the data relating to the claim submission (e.g. using entrymethod analysis module 330C of FIG. 3 ) to determine a time of day, aclaim type, and a method of entry, which may be recognized as riskfeatures linked to varying levels of risk for an interview. The dataassociated with the one or more features may then be stored inshort-term history. In one embodiment, the data associated with the oneor more features may be data that is determined using an AI model. Forexample, an AI model may be trained using a learning algorithm topredict a level of risk associated with the one or more risk features.

At step S406, the server computer may retrieve the user's history. Inone embodiment, the server computer may retrieve the user's history froma database. For example, the server computer may query aggregate datadatabase 230 of FIG. 2 for data linked to the user and/or the userdevice. This may be done based on the user device data collected at stepS403 (e.g. querying a database for data linked to a device ID or IPaddress). Examples of user history that may be retrieved may be data forprevious claim submissions made by the user, identified instances offraud linked to the user device, or any other data that may beassociated with the user's behavior. The user's history may then bestored in short-term history. In one embodiment, the user's history mayinclude risk data associated with the information relating to the user.For example, the user's history may include a level of risk associatedwith communication factors of the user (e.g. IP address, device ID,telephone number, etc.). In an embodiment, the associated risk may bedetermined from index tables. The index tables may comprise risk indexesfor various communication factors of the user, as predicted by an AImodel, such as through a boosted tree or neural network. The riskindexes may be predicted in real-time from data in a transactionprocessing network (e.g. VisaNet) along with external data relating toIP address behavior that may be aggregated from other sources.

At step S407, the server computer may calculate an initial score for theinterview based on data stored in short-term history. The initial scoremay be determined based on at least the feature data and user historycollected at steps S405 and S406 respectively. The initial score may bedetermined based on the predictiveness of the feature data, as learnedthrough training. A learning algorithm may be used to score thepredictiveness of features that may later be present in data, and theoutput associated with the features may be predicted based on thelearned scores. For example, gradient boosting may be used to populatean index table or decision tree, in which various combinations offeature data may each be linked to different interview scores. In oneembodiment, a risk model for assessing the likelihood that a submittedclaim is fraudulent may be used to generate a risk score for theinterview. For example, an ensemble learning method such as a randomdecision forest may be used to generate regression trees for predictingthe probability of fraud associated with an IP address and inputted usertext. When data is received in a claim submission, the IP address andinputted text associated with the submitted claim may be compared to theregression tree in order to determine a risk score, scoring theprobability of a positive identification of fraud for the claim. Forexample, the user may have entered text that is an exact duplicate froma previous interview that was determined to be high risk, which mayresult in a high risk score (e.g. risk score of 90 on a scale of 0 to100) due its high correlation to a known instance of fraud.

At step S408, the server computer may determine, based on the initialscore, if a live interview with a human interviewer should be initiated.In one embodiment, the live interview may be initiated if the initialscore is determined to indicate that the data relating to the claimsubmission has a high probability of containing 100% accurateinformation. For example, a user may submit a request for customersupport that may be determined to have a low risk score (i.e. a genuineclaim submission) and may thus warrant the initiation of a liveinterview with a human representative that may assist the user. In anembodiment, a live interview with a human representative may beinitiated if a score for the interview drops below a predeterminedthreshold (e.g. a risk score below 50).

At step S409, the server computer may update an AI model for determiningan interview script. In one embodiment, the AI model may be updated suchthat the feature data and user history may be correlated. For example,the user history may contain data that has been identified as fraudulentand one or more features of the text input by the user may then becorrelated to fraud in the AI model. This may affect predictions made bythe AI model during an interview. For example, if the same text isrepeated in subsequent claim submissions then the subsequent claimsubmissions may be determined to be associated with fraud. In anotherembodiment, the AI model may be updated such that the one or morefeatures and the user history may be associated with higher risk. Forexample, the one or more features may cause the interview to receive ahigh risk score, which may correlate the user's history (e.g. user ID,device ID, IP address etc.) with higher risk in the AI model.

At step S410, the server computer may query a graph database for a graphbased on a prediction made by the AI model. The graph may be aconceptual graph relating concepts to predicted levels of risk. In oneembodiment, the AI model may receive the initial score for theinterview, initial data entered by the user (i.e. text), the one or morefeatures, and the user history as inputs for predicting an appropriategraph query. In one embodiment, a graph query may be expressed in commonLogic interchange format. For example, a graph query related to‘Claimant 123’ reporting a purchased item that was not received may beexpressed as, (exists ((x Report) (y Fraud-Not received)) (and (claimant123) (agent x 123) (type x y))). In one embodiment, the graph that isqueried and retrieved from the graph database may be a conceptual graphfor forming a sentence. In another embodiment, the server computer maysend the queried graph to the interviewer computer so an interviewscript may be determined. In yet another embodiment, the queried graphmay be used to identify probability of fraud. For example, anoptimization technique for finding shortest paths in a graph (e.g. antcolony optimization) may be used to find a path within a graph thatlinks concepts to a positive indication of fraud. The probability offraud may be a function of the complexity of a claim, the cost of thepath linking detected concepts to fraud (i.e. total edge weight of thepath in a topological graph), and various factors relating to thepotential monetization or payout of a claim. For example, theprobability of fraud may be calculated as: ‘Prob Fraud=f(Claimscomplexity, Path Cost, Detection Easy, Pay out amount)’. Moreinformation regarding using ant colony optimization to find optimalpaths in a graph may be found at:

-   -   C. Blum, “Ant colony optimization: Introduction and recent        trends”, Phys. Life Reviews, vol. 2, pp. 353-373, 2005.

At step S411, the interview script may be determined from the graphretrieved from the graph database. The graph may be a conceptual graphthat may be used to form one or more responses (i.e. sentences) that maybe used in the interview. For example, a conceptual graph may beretrieved that contains the words “location” “user” “live,” which may beused to form the sentence, “Where do you live?” In one embodiment, theinterview script may be determined by the interviewer computer or anagent operating the interviewer computer. In another embodiment,information for the queried graph may be stored in a semantic decisiontable. The semantic decision table may contain logic for linking apredicted outcome made by the AI to facts or lexons. For example, the AImodel may predict an outcome of ‘high risk’ for the interview, which maybe linked in a semantic decision table to words or phrases that may belinked to a follow-up question for asking someone where they live.

At step S412, a response from the interview script may be sent to theuser and/or user device. In one embodiment, the response may be sent tothe user device from the interviewer computer in a message. For example,the response may be sent to the user device over the internet in theform of a message containing text that the user can read (e.g. in a textmessage or through HTML). In another embodiment, the response may besent to the user from an operator of the interviewer computer. Forexample, the operator may read the determined interview script andcommunicate a response from the interview script to the user over thephone.

At step S413, the user may receive the response from the interviewscript and may enter additional data. The additional data may beadditional data relating to the claim submission. For example, the usermay receive a question from the interview script and may generate ananswer by inputting text into the user device. The text may beadditional information that may be used to determine if the claim beingsubmitted is fraudulent or not. The additional data may be sent to theinterviewer computer and then forwarded to the server computer.

At step S414, the server computer may receive the additional data andmay update the short-term history. Updating the short-term history maycomprise storing the additional data as well as data relating to one ormore features recognized from the additional data. For example, theadditional data may comprise text that has been copy and pasted, whichmay be a risk feature that can be used to calculate a subsequent riskscore for the interview. A record of the risk feature being present inthe user's response, as well as the response itself, may be added to theshort-term history alongside the short-term data stored at step S404.

At step S415, the server computer may calculate a subsequent score forthe interview. The subsequent score may be determined using theadditional data entered by the user, one or more features associatedwith the additional data, and/or any other data stored in the short-termhistory (e.g. data stored at step S404). For example, the servercomputer may determine a subsequent risk score for the interview basedon the text of the user's response to a follow-up question and based onthe method of entry for the text.

At step S416, the server computer may determine if the score for theinterview has changed. A score may remain unchanged, for example, if auser has yet to respond to a follow-up question from the interviewscript. If the score for the interview has not changed (e.g. user hasnot responded yet), then the server computer may perform step S417.Otherwise, the server computer may return to performing step S408 in theprocess. The server computer may continue to update the model, performgraph queries, and determine scripts, and the user may continueresponding to affect the score for the interview until either theinterview is terminated or until it is determined that a live interviewshould be initiated (i.e. performing steps S409 though S416 until eitherthe answer to S408 or S417 is “yes”). For example, if the risk score ofthe interview progressively increases with each follow-up question anduser response, then follow-up questions may continually be sent to theuser until either the user quits the interview or begins to enteraccurate information that lowers the risk score. If the risk score dropsbelow a threshold, it may be determined that the information submittedis genuine and a live interview may be initiated.

If at step S416 it is determined that the score for the interview hasnot changed, step S417 may be performed, in which the server computermay determine if the user has terminated the interview. This may bedone, for example, checking if a connection still exists between theuser and the interviewer computer. In another example, the interviewercomputer may send the user device a message containing text for aquestion such as, “are you still there?” The user may respond to thequestion, thereby indicating that the interview has not been terminated.If it is determined that the user has terminated the interview, then theserver computer may perform step S420 in which the interview script isterminated and the session is ended. Otherwise, the server computer mayreturn to step S413, in which the server computer waits for the user toenter additional data. The time it takes for the user to respond may beidentified as a feature (i.e. for making predictions using the AI model)and may be used to update short-term history.

If at step S408 it is determined that a live interview should beinitiated, then step S418 may be performed. In one embodiment, a liveinterview may be initiated if the score for the interview drops below apredetermined threshold (e.g. low risk score, low inaccuracy score,etc.) At step S418, a live agent may be requested. The live agent may berequested, by the server computer, by determining an address of a liveagent. For example, the server computer may search through a list of IPaddresses for computers operated by live agents, and may broadcast, tothe computers, a message indicating that a user needs to be connected toa live agent. In another example, the server computer may search througha list of telephone numbers at which a live agent may be contacted, andmay attempt to reach the agents through telecommunications.

At step S419, the server computer may route the user device to a liveagent. For example, a live agent may receive a queue of users for whicha live agent has been requested, and the live agent may select one ofthe users from the queue. The server computer may then route the userdevice to the live agent's computer or telecommunications device. Thelive agent may further receive the user's information. For example, theinitial data entered by the user, the user's history, and otherinformation relating to the interview between the user and theinterviewer computer may be sent and displayed to the live agent.

At step S420, the interview script may be terminated. For example, theuser may either have been successfully routed to a live agent whom theymay continue the interview with, or the user may have decided to quitthe interview. According to embodiments of the invention, the interviewmay be conducted such that genuine users may be connected to a liveagent who may process a filed claim, while high risk or disingenuoususers may continue to receive questions from the interview script untilthey quit or submit accurate information.

FIG. 5 shows a flowchart for a method for generating a dynamic interviewscript according to an embodiment of the invention. Steps S501 throughS511 may provide a list of steps performed by a system comprising a userdevice of a user, an interviewer computer, and a server computer (e.g.system 200 of FIG. 2 ).

At step S501, user input, user information, and features thereof may bereceived in data. The data may be received from a user device, and maybe data relating to a claims submission. For example, the data mayinclude a text description of a claim that a user wishes to file. Thefeatures of the user input and user information may be risk features fora potentially fraudulent claim submission such as a time of day, a claimtype, and a method of entry.

At step S502, the data may be stored in short-term history. Theshort-term history may be a database storing data from which an AI modelmay use to make predictions about the interview. For example, an AImodel may be trained to make a prediction based on one or more featuresof data received in an interview such as the time of day, the claimtype, the method of entry, and/or the time it takes a user to respond.The prediction may be an assessed risk of the interview based on thefeatures, which may further be used in combination with user history todetermine a risk score.

At step S503, data associated with the features of the data may beretrieved. The data associated with the features may be data that ispredicted by an AI model. The AI model may be a predictive model thattakes in data from short-term history as input and outputs a prediction.For example, the AI model may be trained to score features that, whendetected in data received in a claim submission, may be used to predicta level of riskiness or risk data associated with the claim submission.Examples of risk features for a claims submission may include a claimtype, a claim complexity, a payout method, a monetization potential fora claim, and a repeatability of a submitted claim or text thereof.

At step S504, data associated with the user's information may beretrieved. For example, recorded instances of fraud associated with theuser's name, account number, or user device may be retrieved from anaggregate database. The data associated with the user's information maybe data that may be used as an input to a risk model or risk engine inorder to generate a risk score. Other examples of risk data associatedwith a user may include a prior claims history, location of user device,location of reported incident, a time of day, a day of the week for aclaim submission, a tendency to report higher payout claims, and/or ahigher tendency to claim more serious health conditions.

At step S505, an initial score for the interview may be generated. Theinitial score may be any value that may be used to determine anappropriate response for a given situation (i.e. for a particularinterview being conducted). In one embodiment, the initial score may bea risk score assessing the risk of inaccurate or false information beingpresented in the interview. In another embodiment, the initial score maybe a mood score (e.g. a mood score that assesses the frustration levelof the user).

At step S506, an interview script may be determined based on the initialscore for the interview. The interview script may be determined byperforming a graph query, in which a conceptual graph of words andphrases appropriate for the current conversation (i.e. based on the userinputted text and initial score) may be retrieved. The concepts of theconceptual graph may comprise interrelated concepts that may be used todescribe an interview, as provided by logic of a semantic decisiontable. For example, the semantic decision table may comprise a rulestating that for high risk interviews involving cardholder ‘Rick’claiming a lost/stolen credit card, ‘interview script A: HighRisk’should be initiated. In a topological graph (e.g. concept graph), inputnodes for ‘high risk’, ‘Cardholder: Rick’, and ‘lost/stolen’ may belinked to one another by edges and may further be linked to an outputnode for an interview script. In one embodiment, an appropriateinterview script may be determined by finding optimal paths in a graphfor linking nodes (i.e. interview concepts) to an interview script thatcontains questions that lead to greater effect on an interview score(e.g. questions that provoke responses that greatly increase or decreasea risk score). In one embodiment, optimal paths may be found using anant colony optimization algorithm.

At step S507, a question from the determined interview script may beprovided to the user. The question may be a follow-up question to theuser's inputs that may assist in clarifying the accuracy of information(e.g. the accuracy of information in a claim submission). The user may,in some instances, willingly give false information (such as in the caseof fraud) or may have accidently entered information incorrectly. Ineither case, the follow-up question may be a question that may changethe initial score for the interview to further drive decisions and reachan optimal outcome (e.g. only allowing genuine information to beincluded in a processed claim or deterring criminal actors from gamingthe system).

At step S508, a response from the user may be received. The userresponse may contain one or more features, which may include method ofentry, and/or the time it takes the user to respond. The AI model mayfurther use the one or more features to make a prediction, such as anassessed risk, which may be data that is associated with (i.e. linkedto) the one or more features (e.g. as stored in a table or tree). Theresponse from the user and the data associated with the one or morefeatures may be stored in short-term history along with the data storedat step S502.

At step S509, a subsequent score may be generated based on the user'sresponse and the data stored in short-term history. The subsequent scoremay reflect the current state of the interview's assessed level ofaccuracy or riskiness. For example, the user response may contain aclarification or revision of mistakenly entered information, which maycause a second risk score to be lower than the initial risk score forthe interview.

At step S510, the interview script may be updated based on at least thesecond score. For example, the second score may result in a differentgraph query, and may contain different concepts for determining anappropriate interview script. The updated interview script may containnew questions or concepts that may be used to drive the interviewtowards a desired direction (e.g. may contain questions or responsesthat may lower the user's level of frustration).

At step S511, a second question in the interview script may be providedbased on the second score. In one embodiment the second question may beprovided by the server computer to the interviewer computer. In anotherembodiment, the second question may be selected from the interviewscript by the interviewer computer or operator thereof. For example, theinterview script may contain concepts of a conceptual graph that may beused by an operator of the interviewer computer to ask the question,“excellent, can you wait one moment while we look up your information?”

According to embodiments of the invention, subsequent scores for theinterview may be generated based on user responses. The subsequentscores may be used to update the interview script, so that the user maybe provided follow-up questions that allow them to clarify information.The subsequent scores may also be used to determine that the user haswillingly given false information, which may then result in a continuousinterview script in which the user may receive question after questionuntil he or she terminates the interview.

FIG. 6 shows a depiction of a data flow diagram for an automated claimsubmission system according to an embodiment of the invention. Data flow600 may comprise a plurality of databases such as graph database 603,short-term history 608, and aggregate data database 609, which may beupdated and accessed in conjunction with data such as real-time patternanalysis 601, path determination 602, claims request 604, user interface605, user response 606, risk models engine 607, fraud data 610,real-time data feeds 611, other outside data 612, and model buildenvironment 613.

Real-time pattern analysis 601 may comprise instructions for analyzingpatterns of user response 606 in relation to path determination 602. Forexample, a user may submit a response to a question during an interviewthat comprises specific text, and the text may then be used to determinepaths in a graph in real-time using an AI model. The path determined inpath determination 602 may be a path in a graph queried from graphdatabase 603, such as a conceptual graph relating words or phrases(concepts) to specific user responses and features thereof. Paths may bedetermined using an optimization technique that finds shortest paths ina graph (e.g. ant colony optimization).

A determined path may be a path that connects nodes for concepts relatedto a claims submission to nodes indicating a probability of fraud. Forexample, the queried graph may be a topological graph in which inputnodes relating to a Medicaid claim with a significantly high payout arehighly connected to an output node for fraud.

In one embodiment, real-time learning may be performed by creatingprofiles for various claim signatures. For example, a K-means model forclustering different risk factors together may be continually rebuilt asnew data is received. The clusters may correspond to various profiles ofclaim signatures that are associated with specific levels of risk. Anincoming claim submission may be grouped into a cluster, which maycomprise different claim signature risk factors as the interviewprogresses. For example, a change in a cluster's claim signature riskfactor may be triggered if a high-level of drift between clusters (i.e.a significant difference in risk scores between closely relatedclusters) is detected.

User Interface 605 may be a graphical user interface in which a user maysubmit a claims request 604 as well as submit a user response 606. Forexample, user interface 605 may be a window that may be displayed on auser device that accepts text from a user, which may then be submittedto a server computer to determine a path in path determination 602. Theuser interface may further comprise logic that may be used to performmethod of entry analysis. For example, the user interface may comprisecode for detecting pauses between typed inputs and detecting if wordshave been copy and pasted. The method of entry analysis may then furtheraffect a determined or updated interview script. For example, the methodof entry of a user response may be associated with one or more riskfeatures from which an AI model may use to generate a risk score andpredict an appropriate graph query. In one embodiment, packages ofinformation on relating to how data is entered may be a client-basedreal-time monitoring tool developed using JavaScript. Data received froma user response may be stored in short-term history, so that features ofthe data may be used as inputs to a predictive model (i.e. AI model).

Risk models engine 607 may comprise instructions for determining a riskscore assessing the risk associated with information of, or entered by,a user. The risk associated with information of, or entered by, the usermay be determined from user information and user entered text stored inshort-term history (e.g. from user response 606), and may be determinedusing an AI model.

An AI model may be created in model build environment 613, usingtraining data from aggregate data database 609. For example, a learningalgorithm may use fraud data 610, real-time data feeds 611, and otheroutside data 612 as training data, and the AI model may be trained torecognize patterns in user responses for making predictions. Fraud data610 and real-time data feeds 611 may be data collected from atransaction network (e.g. VisaNet), in which transaction data andcharacteristic thereof may be recorded. For example, fraud data 610 maycomprise data for flagging fraudulent transactions, and real-time datafeeds 611 may comprise data for transactions between consumers andmerchants as they occur. Other outside data 612 may comprise any otherdata that may be aggregated to facilitate training, such as social-mediadata or census data.

Predictions made by the created AI model may be used to determine riskassociated with an interview as well as determine paths in a graph sothat an appropriate interview script may be generated. During aninterview, user responses may further be stored in aggregate datadatabase 609, such that the AI model may be updated. The AI model maythen recognize patterns, and determine an optimal path within a graph ofgraph database 603, that may be used to determine or update an interviewscript. Thus, the interview script may be dynamically generated in a waythat does not allow a user to easily predict follow-up questions to eachuser response 606.

FIG. 7 shows a data flow diagram for building an artificial intelligencemodel for generating an interview script according to an embodiment ofthe invention. Data flow 700 may comprise one or more databasesincluding graph database 701, simulated short-term history, aggregatedata database 707, and modified graph database 709, which may beaccessed and updated in conjunction with edge file generation 702,real-time simulator 703, interview outcome stats 705, user responsestats 706, model build graph 708, learning algorithm 710, and externaldata 711.

According to embodiments of the invention, the elements of system 700may be controlled and/or utilized by a server computer, such as servercomputer 300A of FIG. 3 . The server computer may begin building an AImodel by accessing graph database 701. The graph database 701 maycomprise a plurality of topological graphs. The topological graphs maybe conceptual graphs, wherein concepts relating to an interview scriptmay be connected by edges. For example, conceptual graphs may comprisenodes for a user ID, a type of claim, and a risk score, which may beconnected to various words or phrases that may be included in aninterview script. An example of a conceptual graph is further describedin FIG. 8 below.

The server computer may begin generating edges for graphs in graphdatabase 701 in edge file generation 702. Edge file generation maycomprise instructions for connecting related nodes in a topologicalgraph. For example, edge file generation may comprise code for linkingnodes for various concepts or data relating to a claims submissiontogether in order to build a conceptual graph that may be used todetermine an interview script.

Edges in a graph may be generated based on data relating to aninterview, such as data relating to a claims submission. The data may becollected from aggregate data database 707, in which interview data andexternal data 711 may be aggregated. The collected data may be used todetermine interview outcome stats 705 and user response stats 706. Forexample, aggregate data database may comprise data relating to thewhether or not an interview ended in an identification of fraud, anidentification of a genuinely submitted claim, or any other possibleoutcome for an interview, and may further comprise data relating toinformation about user responses, such as whether or not a user responseincreased or decreased a risk score for the interview.

The interview outcome stats 705 and user response stats 706 may be fedinto a real-time simulator, in which information collected frominterviews may be used to generate a sequence graph. The sequence graphmay be a graph in which sequences of nodes may be truncated to formrules. Each rule may be a path in a graph, which links nodes relating tointerview data to nodes for concepts that may be used to determine anappropriate interview script. For example, a sequence graph may comprisea path of nodes in which nodes for ‘cardholder: Rick’, ‘high risk’, and‘lost/stolen card’ may be linked together to ‘Interview Script A.’ Datafor the sequence graph may be stored in simulated short-term history704, so that a sequence path may be used to simulate how a potentialinterview may end.

If a generated sequence graph comprises a new rule, then the sequencepath for the new rule may end. The new rule may be used to build an AImodel for determining a dynamic interview script. Model build graph 708may build an AI model based on the new rules determined from real-timesimulator 703, as well as data from modified graph database 709, andlearning algorithm 710. Learning algorithm 710 may comprise a learningalgorithm, in which features of an interview may be used to determineoptimal paths in a graph. Optimal paths for the graphs may be recordedin graph database 709.

According to embodiments of the invention, an AI model may be used todetermine an interview script that may prevent the inclusion ofinaccurate information in a processed claim or that may deter a criminalactor from a gaming a claim submission system. For example, the AI modelgenerated using system 700 may be an AI model that detects features in aclaims submission such as a claim type, features related to the methodof entry of a claim, and risk levels associated with a user in order topredict an interview script that may assist in identifying if a user islying. The AI model may predict the appropriate interview script byidentifying a path in a queried graph, in which claim data in aninterview may be linked to a specific set of words or phrases based onlearned rules.

FIG. 8 shows an example of a path of nodes in a concept graph accordingto an embodiment of the invention. According to embodiments of theinvention, the graph may be a conceptual graph for determining aninterview script, and may be queried from a graph database during adynamic interview process (e.g. during a claims submission). Anadvantage of using a topological graph, such as a conceptual graph, todetermine an interview script is that relationships between concepts(e.g. relationships between claim data and risk levels) can be learnedand re-evaluated, and optimal paths within the graph may be determinedfor reaching a desired outcome (e.g. for preventing inaccurateinformation from being submitted in a processed claim). This differsfrom determining interview questions from a hardcoded table or tree, inthat better suited questions may be determined as learning occurs, andas connections are evaluated.

Example path 800 may comprise a set of nodes such as node 801:

‘Cardholder: Rick 801’, node 802: ‘claims’, node 803: ‘Lost/Stolen’,node 804: ‘Starts’, node 805: ‘Script:A.Point1’, node 806: ‘initiate’,node 807: ‘Script:A.Point1.HighRisk’, node 808: ‘HighRisk’, node 809:‘Is’, and node 810: ‘Initiate’. In the conceptual graph shown, nodes forconcepts may be shown as rectangles, and nodes for conceptual relationsthat link concepts may be shown as circles.

During an interview, data associated with a user and data associatedwith a claim submitted by the user may be received by a server computer(e.g. server computer 300A of FIG. 3 ). The data may be used to performa graph query, in which a conceptual graph comprising nodes related tothe data may be retrieved. From the conceptual graph, an appropriateinterview script for the current state of the interview may bedetermined.

For example, during an interview, a user may submit a claim for a stolencredit card. An AI model comprising a tiered modeling architecture ofseparate sub-modules may be used to determine data associated with theuser and the entered claim. For example, a neural network may be used todetermine communication risk factors associated with the user's IPaddress and device ID, and risk factors associated with data entered bythe user may be determined using risk profiles developed thoughunsupervised learning (e.g. clustering). A risk score for the interviewmay further be determined from the data associated with the user and theentered claim. For example, a risk scoring engine running on the servercomputer may take a user's stored history and risk features associatedwith a claim as inputs, and may output a risk score based on riskmodeling performed in a transaction processing network.

Based on the risk score and data associated with the user and submittedclaim, a graph query may be performed by the server computer. Thequeried graph may be a conceptual graph comprising example path 800.Example path 800 may then be used to initiate an interview script. For aclaim comprising ‘cardholder:ABC’ claiming a card being ‘lost or stolen’may cause ‘Script:A.Point1’ to be initiated. ‘Script:A.Point1’ may be aninitial interview script that may comprise one or more responses orfollow-up questions for receiving additional information from the user.For example, the initial interview script may comprise the following:“Please Enter: Time of day and Exact Location of Incident.”

Responses from the interview script may be sent from an interviewercomputer to the user's device, and the user may then enter a userresponses comprising additional information (e.g. user enters the timeof day and exact location of the reported incident). The additionalinformation may then change the calculated risk score for the interview,and may result in an updated interview script. For example, the user mayenter a time of day and exact location that was previously used in afraudulent claim submission, which may cause the risk score to change to‘High Risk’. The interview script may then be updated, and may compriseadditional follow-up responses that may extract further information fromthe user. For example, the updated interview script may then comprisethe following questions: “Did you notify the police? If so, entercontact information; Why were you at the location?; Did you notifysurrounding businesses?”

According to embodiments of the invention, interview scripts determinedusing the AI model may be generated such that only accurate informationmay be included in a processed claim. For example, an interviewdetermining to be associated with a high risk of inaccurate or falseinformation may result in a continuous interview script containing moreaggressive questioning. If the risk score for the interview continues toincrease or remain at high levels, then follow-up questions may becontinually sent to the user until he or she quits the interview. If theuser begins to clarify information such that the information is accurateor if the user begins to act in a manner that is determined to begenuine (as recognized through learned patterns), the risk score for theinterview may lower. If the risk score decreases below a predefinedthreshold, then the user may be routed to a live interview, wherein ahuman representative may process the claim filed by the user.

Embodiments of the invention provide a number of technical advantagesover prior art. Prior methods for generating an interview scriptinvolved merely generating static scripts for each situation. In priormethods, each script or response is mapped as a static decision, andcriminal actors are able to game the system by learning what type ofresponses resulted in a set of decisions that lead to their desiredoutcome (e.g. processing of a fraudulent claim). In contrast,embodiments of the present invention utilize an AI model that maycontinuously learn during an interview, and may be used to dynamicallychange an interview script in ways that cannot be easily predicted by auser. Furthermore, in other claims submission processes, human error onboth the part of the user and the interviewer may result in undesiredresults. For example, a confusing user interface may cause a user toincorrectly enter information, or a poorly trained interviewer mayinteract with a user in a manner that frustrates the user. Embodimentsof the invention remedy these issues by constantly evaluating the stateof an interview via scoring and modeling, which may provide tailoredquestions for clarifying information and driving an interview towards adesired outcome.

Any of the computer systems mentioned herein may utilize any suitablenumber of subsystems. Examples of such subsystems are shown in FIG. 2 incomputer apparatus 202, 210, and 220. In some embodiments, a computersystem includes a single computer apparatus, where the subsystems can bethe components of the computer apparatus. In other embodiments, acomputer system can include multiple computer apparatuses, each being asubsystem, with internal components. A computer system can includedesktop and laptop computers, tablets, mobile phones and other mobiledevices.

The subsystems shown in FIG. 2 may be interconnected via a system bus.Additional subsystems such as a printer, keyboard, storage device(s),monitor, which is coupled to display adapter, and others are shown.Peripherals and input/output (I/O) devices, which couple to I/Ocontroller, can be connected to the computer system by any number ofconnections known in the art such as input/output (I/O) port (e.g., USB,FireWire). For example, I/O port or external interface (e.g. Ethernet,Wi-Fi, etc.) can be used to connect computer system to a wide areanetwork such as the Internet, a mouse input device, or a scanner. Theinterconnection via system bus allows the central processor tocommunicate with each subsystem and to control the execution of aplurality of instructions from system memory or the storage device(s)(e.g., a fixed disk, such as a hard drive, or optical disk), as well asthe exchange of information between subsystems. The system memory and/orthe storage device(s) may embody a computer readable medium. Anothersubsystem is a data collection device, such as a camera, microphone,accelerometer, and the like. Any of the data mentioned herein can beoutput from one component to another component and can be output to theuser.

A computer system can include a plurality of the same components orsubsystems, e.g., connected together by external interface or by aninternal interface. In some embodiments, computer systems, subsystem, orapparatuses can communicate over a network. In such instances, onecomputer can be considered a client and another computer a server, whereeach can be part of a same computer system. A client and a server caneach include multiple systems, subsystems, or components.

Aspects of embodiments can be implemented in the form of control logicusing hardware (e.g. an application specific integrated circuit or fieldprogrammable gate array) and/or using computer software with a generallyprogrammable processor in a modular or integrated manner. As usedherein, a processor includes a single-core processor, multi-coreprocessor on a same integrated chip, or multiple processing units on asingle circuit board or networked. Based on the disclosure and teachingsprovided herein, a person of ordinary skill in the art will know andappreciate other ways and/or methods to implement embodiments of thepresent invention using hardware and a combination of hardware andsoftware.

Any of the software components or functions described in thisapplication may be implemented as software code to be executed by aprocessor using any suitable computer language such as, for example,Java, C, C++, C#, Objective-C, Swift, or scripting language such as Perlor Python using, for example, conventional or object-orientedtechniques. The software code may be stored as a series of instructionsor commands on a computer readable medium for storage and/ortransmission. A suitable non-transitory computer readable medium caninclude random access memory (RAM), a read only memory (ROM), a magneticmedium such as a hard-drive or a floppy disk, or an optical medium suchas a compact disk (CD) or DVD (digital versatile disk), flash memory,and the like. The computer readable medium may be any combination ofsuch storage or transmission devices.

Such programs may also be encoded and transmitted using carrier signalsadapted for transmission via wired, optical, and/or wireless networksconforming to a variety of protocols, including the Internet. As such, acomputer readable medium may be created using a data signal encoded withsuch programs. Computer readable media encoded with the program code maybe packaged with a compatible device or provided separately from otherdevices (e.g., via Internet download). Any such computer readable mediummay reside on or within a single computer product (e.g. a hard drive, aCD, or an entire computer system), and may be present on or withindifferent computer products within a system or network. A computersystem may include a monitor, printer, or other suitable display forproviding any of the results mentioned herein to a user.

Any of the methods described herein may be totally or partiallyperformed with a computer system including one or more processors, whichcan be configured to perform the steps. Thus, embodiments can bedirected to computer systems configured to perform the steps of any ofthe methods described herein, potentially with different componentsperforming a respective steps or a respective group of steps. Althoughpresented as numbered steps, steps of methods herein can be performed ata same time or in a different order. Additionally, portions of thesesteps may be used with portions of other steps from other methods. Also,all or portions of a step may be optional. Additionally, any of thesteps of any of the methods can be performed with modules, units,circuits, or other means for performing these steps.

The specific details of particular embodiments may be combined in anysuitable manner without departing from the spirit and scope ofembodiments of the invention. However, other embodiments of theinvention may be directed to specific embodiments relating to eachindividual aspect, or specific combinations of these individual aspects.

The above description of example embodiments of the invention has beenpresented for the purposes of illustration and description. It is notintended to be exhaustive or to limit the invention to the precise formdescribed, and many modifications and variations are possible in lightof the teaching above.

A recitation of “a”, “an” or “the” is intended to mean “one or more”unless specifically indicated to the contrary. The use of “or” isintended to mean an “inclusive or,” and not an “exclusive or” unlessspecifically indicated to the contrary. Reference to a “first” componentdoes not necessarily require that a second component be provided.Moreover reference to a “first” or a “second” component does not limitthe referenced component to a particular location unless expresslystated.

All patents, patent applications, publications, and descriptionsmentioned herein are incorporated by reference in their entirety for allpurposes. None is admitted to be prior art.

1.-20. (canceled)
 21. A method comprising: a) receiving, by a firstcomputer, data relating to a claim submission from a second computer,wherein the data relating to the claim submission includes claimssubmission data input by a user, information relating to the user, andone or more features, wherein the claim submission is a request forsomething that is owed to the user; b) storing, by the first computer,the data relating to the claim submission in a first database; c)retrieving, by the first computer, from the first database, dataassociated with the one or more features, wherein the data associatedwith the one or more features is determined from an artificialintelligence model, wherein the artificial intelligence model is builtby accessing a graph database to obtain a plurality of topologicalgraphs, generating a plurality of edges associated with nodes in thetopological graphs to build a conceptual graph, and inputting interviewoutcome data, user response data, and the conceptual graph into a realtime simulator to build one or more sequence graphs, the one or moresequence graphs used to generate the artificial intelligence model; d)retrieving, by the first computer, from a second database, dataassociated with the information relating to the user; e) generating, bythe first computer, a first score based on the data input by the user,the data associated with the one or more features and the dataassociated with the information relating to the user, the first scoreevaluating whether the data input by the user is fraudulent; f)determining, by the first computer, an interview script based at leastupon the first score; g) providing, by the first computer, a firstquestion in the interview script to the second computer; h) receiving,by the first computer, from the second computer, a response to the firstquestion i) generating, by the first computer, a second score based atleast upon the data in the first database and the response to the firstquestion, and then updating, by the first computer, the artificialintelligence model using at least the second score and the response tothe first question; j) updating, by the first computer, the interviewscript based at least upon the second score and the updated artificialintelligence model; and k) providing, by the first computer, a secondquestion in the updated interview script to the second computer, thesecond question based at least in part upon the second score.
 22. Themethod of claim 21 further comprising: l) receiving, by the firstcomputer, from the second computer, a response to the second question;m) generating, by the first computer, a third score based at least uponthe data in the first database and the response to the second question;n) updating, by the first computer, the updated interview script basedat least upon the third score to form a subsequently updated interviewscript; and o) providing, by the first computer, a third question in thesubsequently updated interview script to the second computer, the thirdquestion based at least in part upon the third score.
 23. The method ofclaim 22, wherein the one or more features include a time of day, aclaim type, a method of entry, and an average time to respond.
 24. Themethod of claim 22, wherein the one or more features include a time ofday, a claim type, a method of entry, and/or an average time to respond.25. The method of claim 22, wherein the data relating to the claimsubmission is received by the second computer from a user device of theuser, and wherein the information relating to the user includesinformation relating to the user device.
 26. The method of claim 22,wherein the one or more features include features of text inputted bythe user, and wherein the features of the text inputted by the user aredetermined using a natural language parser.
 27. The method of claim 22,wherein the one or more features are risk features relating to a risk ofinaccurate information being included in the data relating to the claimsubmission, and wherein the first score, second score, and third scoreare risk scores.
 28. The method of claim 21, wherein the first computercontinues to provide questions to the second computer if a continuallyupdated score remains above a predetermined value, and wherein the useris routed to a live interview with a human representative if acontinually updated score drops below another predetermined value.
 29. Aserver computer comprising: a network interface; a processor; and anon-transitory computer-readable medium comprising code for instructingthe processor to implement a method, the method comprising: a)receiving, by the server computer, data relating to a claim submissionfrom a client computer, wherein the data relating to the claimsubmission includes claims submission data input by a user, informationrelating to the user, and one or more features, wherein the claimsubmission is a request for something that is owed to the user; b)storing, by the server computer, the data relating to the claimsubmission in a first database; c) retrieving, by the server computer,from the first database, data associated with the one or more features,wherein the data associated with the one or more features is determinedfrom an artificial intelligence model, wherein the artificialintelligence model is built by accessing a graph database to obtain aplurality of topological graphs, generating a plurality of edgesassociated with nodes in the topological graphs to build a conceptualgraph, and inputting interview outcome data, user response data, and theconceptual graph into a real time simulator to build one or moresequence graphs, the one or more sequence graphs used to generate theartificial intelligence model; d) retrieving, by the server computer,from a second database, data associated with the information relating tothe user; e) generating, by the server computer, a first score based onthe data input by the user, the data associated with the one or morefeatures and the data associated with the information relating to theuser, the first score evaluating whether the data input by the user isfraudulent; f) determining, by the server computer, an interview scriptbased at least upon the first score; g) providing, by the servercomputer, a first question in the interview script to the clientcomputer; h) receiving, by the server computer, from the clientcomputer, a response to the first question; i) generating, by the servercomputer, a second score based at least upon the data in the firstdatabase and the response to the first question, and then updating theartificial intelligence model using at least the second score and theresponse to the first question; j) updating, by the server computer, theinterview script based at least upon the second score and the updatedartificial intelligence model; and k) providing, by the server computer,a second question in the updated interview script to the clientcomputer, the second question based at least in part upon the secondscore, wherein the server computer continues to provide questions to theclient computer if a continually updated score remains above apredetermined value, and wherein the user is routed to a live interviewwith a human representative if a continually updated score drops belowanother predetermined value.
 30. The server computer of claim 29,wherein the method further comprises: l) receiving, by the servercomputer, from the client computer, a response to the second question;m) generating, by the server computer, a third score based at least uponthe data in the first database and the response to the second question;n) updating, by the server computer, the updated interview script basedat least upon the third score to form a subsequently updated interviewscript; and o) providing, by the server computer, a third question inthe subsequently updated interview script to the client computer, thethird question based at least in part upon the third score. 31.(canceled)
 32. The server computer of claim 30, wherein the one or morefeatures include a time of day, a claim type, a method of entry, and/oran average time to respond.
 33. The server computer of claim 30, whereinthe data relating to the claim submission is received by the clientcomputer from a user device of the user, and wherein the informationrelating to the user includes information relating to the user device.34. The server computer of claim 30, wherein the one or more featuresinclude features of text inputted by the user, and wherein the featuresof the text inputted by the user are determined using a natural languageparser.
 35. The server computer of claim 30, wherein the one or morefeatures are risk features relating to a risk of inaccurate informationbeing included in the data relating to the claim submission, and whereinthe first score, second score, and third score are risk scores.
 36. Theserver computer of claim 29, wherein the server computer continues toprovide questions to the client computer if a continually updated scoreremains above a predetermined value, and wherein the user is routed to alive interview with a human representative if the continually updatedscore drops below another predetermined value.
 37. A client computercomprising: a network interface; a processor; and a non-transitorycomputer-readable medium comprising code for instructing the processorto implement a method, the method comprising: a) sending, by the clientcomputer, data relating to a claim submission to a server computer,wherein the data relating to the claim submission includes claimssubmission data input by a user, information relating to the user, andone or more features, wherein the claim submission is a request forsomething that is owed to the user; b) receiving, by the clientcomputer, a first question in an interview script from the servercomputer, wherein the interview script is determined based at least upona first score generated based on data determined from an artificialintelligence model using the data relating to the claim submission, thefirst score evaluating whether the data input by the user is fraudulent,wherein the artificial intelligence model is built by accessing a graphdatabase to obtain a plurality of topological graphs, generating aplurality of edges associated with nodes in the topological graphs tobuild a conceptual graph, and inputting interview outcome data, userresponse data, and the conceptual graph into a real time simulator tobuild one or more sequence graphs, the one or more sequence graphs usedto generate the artificial intelligence model; c) sending, by the clientcomputer, the first question in the interview script to a user device ofthe user; d) receiving, by the client computer, a response to the firstquestion; e) forwarding, by the client computer, the response to thefirst question to the server computer, wherein the server computergenerates a second score, updates the artificial intelligence modelusing the response to the first question and the second score, andobtains an updated interview script using the updated artificialintelligence model; and f) receiving, by the client computer, a secondquestion in the updated interview script from the server computer,wherein the second question is determined based at least upon the secondscore generated based at least upon the response to the first question.38. The client computer of claim 37, wherein the first and second scoresare generated based on data associated with one or more featuresincluded in the data relating to the claim submission.
 39. The clientcomputer of claim 38, wherein the method further comprises: g) sending,by the client computer, to the server computer, a response to the secondquestion; and h) receiving, by the client computer, a third question ina subsequently updated interview script based on the updated interviewscript from the server computer, the third question based at least inpart upon a third score generated based at least upon the response tothe second question.
 40. The method of claim 37, wherein the servercomputer continues to provide questions to the client computer if acontinually updated score remains above a predetermined value, andwherein the user is routed to a live interview with a humanrepresentative if a continually updated score drops below anotherpredetermined value.